Method for transmitting and receiving image, receiving device, and image storage device

ABSTRACT

A technique is provided for reducing the size of each of images included in an image signal and generating a high-resolution image with minimized degradation in the image quality from the reduced images. Alias components and motion information that are included in the image signal having the reduced images are used for conversion of the images included in the image signal into a high-resolution image. Low pass filtering is performed on a frequency component in the direction of a motion included in the image signal and a frequency component in a direction other than the direction of the motion. The cut-off frequency of the low pass filter in the direction other than the direction of the motion is lower than the cut-off frequency of the low pass filter in the direction of the motion.

CLAIM OF PRIORITY

The present application claims priority from Japanese patent applicationserial No. JP 2007-244643, filed on Sep. 21, 2007, the content of whichis hereby incorporated by reference into this application.

BACKGROUND OF THE INVENTION

(1) Field of the Invention

The present invention relates to a technique for reducing the size of animage and enlarging the image after the reduction.

(2) Description of the Related Art

It is known that JP-A-2005-348320 discloses a typical one of techniquesfor reducing the size of an image and restoring the image after thereduction.

It is also known that Non-Patent Document 1 (Shin Aoki: “SuperResolution Processing By Plural Number of Lower Resolution Images”,Ricoh Technical Report, pp. 19-25, No. 24, November, 1998) discloses asuper resolution technique as a technique for generating an image havingincreased resolution from plural images including alias components.

SUMMARY OF THE INVENTION

The technique disclosed in Non-Patent Document 1 is capable ofincreasing resolution of an image only in the direction in which asubject moves. In the conventional technique, although image resolutionin the horizontal direction in which the subject moves can be increased,image resolution in the vertical direction cannot be increased, as shownin FIG. 5.

In such a technique for increasing resolution of an image as describedin Non-Patent Document 1, which restores the image after the reductionin the size of the image, when the size of the image is reduced by pixelsubsampling, an alias component is included in the reduced image in animage reduction process. In this case, the alias component remains asnoise in a frequency component (of an image signal including the imageobtained after the increase in the resolution) in a direction in whichresolution cannot be increased. For example, the alias component appearsas noise to reduce the quality of the image.

In addition, when a spatially invariant low pass filter is used for areduction in the size of an image, a high-resolution image cannot begenerated even if the super resolution processing technique described inNon-Patent Document 1 is used.

To solve the problem, the following processing is performed according toan aspect of the present invention. That is, the size of each of imagesincluded in an image signal is reduced, and the image signal isconverted into a high-resolution image by using an alias component andinformation on a motion of a subject included in the images. The aliascomponent and the information are included in the image signal havingthe reduced images.

Then, low pass filtering is performed to cut a high frequency componentin the direction of the motion included in the image signal. In thiscase, the cut-off frequency of the low pass filter for the image signalin the direction other than the direction of the motion is lower thanthe cut-off frequency of the low pass filter for the image signal in thedirection of the motion.

According to the aspect of the present invention, a high-resolutionimage can be generated from reduced images, with minimized degradationin the image quality.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features, objects and advantages of the presentinvention will become more apparent from the following description whentaken in conjunction with the accompanying drawings wherein:

FIG. 1 is a diagram showing an example of the configuration of an imagetransmission/reception system according to a first embodiment of thepresent invention;

FIG. 2 is a diagram showing an example of the configuration of an imagereduction unit according to the first embodiment;

FIG. 3 is a diagram showing an example of coefficients of low passfilters according to the first embodiment;

FIG. 4 is a diagram showing another example of the configuration of theimage reduction unit according to the first embodiment;

FIG. 5 is a diagram showing an example of an image of a subject movingin the horizontal direction;

FIG. 6 is a diagram showing an example of the configuration of an imagestorage device according to a second embodiment of the presentinvention;

FIG. 7 is a diagram showing an example of the configuration of an imagetransmission/reception system according to a third embodiment of thepresent invention;

FIG. 8 is a diagram showing an example of the configuration of an imagecorrection unit according to the third embodiment;

FIG. 9 is a diagram showing another example of the configuration of theimage correction unit according to the third embodiment;

FIG. 10 is a diagram showing another example of the configuration of theimage storage device according to a fourth embodiment of the presentinvention;

FIG. 11 is a diagram showing an example of the configuration of an imageresolution increasing unit according to the first embodiment;

FIGS. 12A to 12E are diagrams showing processing for increasingresolution according to the first embodiment;

FIGS. 13A to 13C are diagrams showing the processing for increasingresolution according to the first embodiment;

FIGS. 14A to 14C are diagrams showing the processing for increasingresolution according to the first embodiment;

FIG. 15 is a graph explaining a rate increasing unit according to thefirst embodiment;

FIG. 16 is a graph explaining the rate increasing unit according to thefirst embodiment;

FIGS. 17A and 17B are graphs explaining a phase shift device accordingto the first embodiment;

FIG. 18 is a graph explaining the phase shift device according to thefirst embodiment;

FIGS. 19A to 19D are diagrams explaining a coefficient determinationdevice according to the first embodiment.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

An embodiment of the present invention will be described with referenceto the accompanying drawings.

In the drawings, constitutional elements denoted by the same referencenumeral have the same function(s).

The meaning of the “phase” of an image signal includes the meaning of a“location” in the image signal in the description in the presentspecification and the accompanying drawings.

First Embodiment

Referring to FIG. 1, a transmitting device 100 includes an image signalinput section 1, an image reduction unit 2, encoder 3, a motion detector8, a transmitter 11, a network interface 13, and a motion vectortransformer 15. The image reduction unit 2 receives an image signal fromthe image signal input section 1 and reduces the size of an image(s)included in the image signal. The rate (image reduction rate) of thereduction in the size of the image can be changed based on performanceof an image resolution increasing unit 6 or a demanded quality of theimage to be output. The image resolution increasing unit 6 is providedon the downstream side of the image reduction unit 2.

In the first embodiment described below, the length and width of theimage(s) included in the image signal are reduced by, for example, halfby the image reduction unit 2.

In the above case, the motion detector 8 also receives the image signalfrom the image signal input section 1 and calculates a motion vector(s)included in the image signal. The image reduction unit 2 performs lowpass filtering based on the motion vector(s) calculated by the motiondetector 8. The operations of the image reduction unit 2 and theoperations of the motion detector 8 are described later.

The image signal including the image(s) reduced by the image reductionunit 2 is encoded, for example, by the encoder 3. When the size of theimage(s) is reduced before the encoding, it can be expected that animage compression rate is increased (data to be encoded has a smallamount of bits). Image encoding schemes including MPEG-2 and MPEG-4 havebeen widely known as a method for encoding an image signal in order totransmit and store the image signal. In the present embodiment, thecompression rate of an image signal can be increased by using any of theimage encoding schemes as described above. Any of the image encodingschemes may be used as long as the compression rate of an image signalis increased in the present embodiment.

When the encoder 3 uses a motion vector(s) for encoding an image signal,the encoder 3 performs motion detection to obtain the motion vector(s).The encoder 3 uses the obtained motion vector(s) to encode the imagesignal and then generate a bit stream. In this case, the generated bitstream includes the motion vector(s) and the image signal.

In another example of the configuration of the imagetransmission/receiving system, information on the motion vector(s)calculated by the motion detector 8 based on a motion included in animage signal may be used. In this case, the size of the image includedin the image signal used for the calculation of the motion vector(s) bythe motion detector 8 is different from the size of the image includedin an image signal to be encoded by the encoder 3. The motion vectortransformer 15 changes the number of the motion vectors (generated bythe motion detector 8) and the length of each of the motion vectorsbased on the image reduction rate of the image reduction unit 2 togenerate a motion vector(s) that can be used by the encoder 3. When theencoder 3 uses the motion vector(s) generated by the motion vectortransformer 15, motion detection processing to be performed by theencoder 3 is not necessary. This results in a reduction in the amount ofdata to be processed.

In the transmitting device 100, the image data encoded by the encoder 3is then transmitted from the encoder 3 to the transmitter 11 or to areceiving device 101 through the network interface 13.

When the transmitting device 100 is a device for broadcast station orthe like, for example, the transmitter 11 transmits the image data tothe receiving device 101 by means of a radio wave or the like.

When the transmitting device 100 is an image distribution server, atransmitter for Internet Protocol Television or the like, the networkinterface 13 transmits the image data through a network 4 to thereceiving device 101.

The receiving device 101 includes a receiver 12, a network interface 14,a decoder 5, an image memory 9, the image resolution increasing unit 6,an image signal output section 7, and a display unit 10. In thereceiving device 101, the encoded image data is transmitted from thetransmitter 11 and received by the receiver 12 or is transmitted fromthe network interface 13 and received by the network interface 14.

When the receiving device 101 is a broadcasting receiver or the like,the encoded image data transmitted by means of a radio wave or the likeis received by the receiver 12.

When the receiving device 101 is a client of an image distributionserver, a receiver for Internet Protocol Television or the like, thenetwork interface 14 receives the encoded image data transmitted throughthe network 4.

The decoder 5 decodes the received encoded image signal. The imagesignal decoded by the decoder 5 is stored in the image memory 9. Theimage resolution increasing unit 6 generates an image having increasedresolution from the multiple images included in the image signal storedin the image memory 9. In this case, the image resolution increasingunit 6 performs processing for increasing resolution (processing forconverting the image signal into a high-resolution image) by using analias component(s) and a difference between positions of a subjectincluded in the images. The processing for increasing the resolutionwill be described later.

The image resolution increasing unit 6 transmits the image havingincreased resolution to the display unit 10 or the image signal outputsection 7. The display unit 10 receives the image from the imageresolution increasing unit 6 and displays the image. The image signaloutput section 7 receives the image from the image resolution increasingunit 6 and outputs the image to another device.

Next, the image reduction unit 2 provided in the transmitting device 100according to the first embodiment will be described with reference toFIG. 2.

The image reduction unit 2 has a plurality of low pass filters 21 and aselector 22. The image signal received by the image reduction unit 2 isinput in parallel to the plurality of spatial low pass filters 21 havingrespective characteristics different from each other. Each of the lowpass filters 21 outputs a signal to the selector 22. The selector 22receives the signals from the low pass filters 21 and selects one of thereceived signals. The selector 22 outputs the selected signal.

The image signal received by the image reduction unit 2 is alsotransmitted to the motion detector 8. The motion detector 8 detects amotion vector for each of pixels (pixel values) included in the imagesignal, and outputs information (motion vector information) on themotion vectors to the selector 22. The selector 22 selects one of thesignals output from the low pass filters 21 based on the motion vectorinformation.

The selector 22 subsamples the pixels to reduce the size of the image.In the first embodiment, the selector 22 outputs a pixel value to animage output section 25 for each pixel and each line to reduce thelength and width of the image by half. To reduce the size of the imageat a rate different from that in the abovementioned case, the selector22 may subsample the pixels at a rate different from that in theabovementioned case.

Next, a description will be made of operations of the selector 22 basedon characteristics of the plurality of low pass filters 21. The low passfilters 21 have respective different characteristics of directions andfrequency ranges to be limited. For example, the low pass filter (LPF) 1(which is one of the low pass filters 21) blocks a high frequencycomponent (of the received image signal) in the vertical direction toreduce the frequency range of the image signal by half and does notblock a frequency component (of the received image signal) in thehorizontal direction. The low pass filter (LPF) 2 (which is another oneof the low pass filters 21) blocks a high frequency component (of thereceived image signal) in the horizontal direction to reduce thefrequency range of the image signal by half and does not block afrequency component (of the received image signal) in the verticaldirection.

FIG. 3 shows an example of coefficients of the low pass filters 1 and 2in the above case. The selector 22 operates to select signals outputfrom the low pass filter 1 for pixels representing a subject 51 (shownin FIG. 5) moving only in the horizontal direction. This results fromthe fact that since the image resolution increasing unit 6 increases theresolution by using the difference between positions of a subject,resolution only in the direction in which the subject moves can beincreased. In this case, therefore, resolution in the vertical directionin which the subject does not move is not increased.

The selector 22 according to the first embodiment uses an appropriateone or more of the low pass filters 21 to limit the frequency range of afrequency component (of the received image signal) in a direction inwhich a subject does not move, or a direction perpendicular to adirection in which the subject moves, i.e., the vertical direction,before subsampling the pixels.

In this case, the selector 22 according to the first embodiment does notperform low pass filtering on a frequency component (of the receivedimage signal) in a direction in which the subject moves, i.e., thehorizontal direction before subsampling the pixels. Alternatively, theselector 22 performs low pass filtering on the received image signal tocut a high frequency component (in the vertical direction). In thiscase, the cut-off frequency in the horizontal direction is higher thanthe cut-off frequency in the vertical direction, and a reduction inalias components is small.

Since the selector 22 performs the abovementioned processing, thetransmitting device 100 outputs an image signal including an aliascomponent in the direction which a subject moves and the reduced amountof alias components in a direction in which no subject moves. Inaddition, after the size of the image(s) is reduced, data on the reducedimage(s) is transmitted by the transmitting device 100. This results ina reduction in the amount of data to be transmitted. The transmittingdevice 100 is therefore capable of outputting an image signal includinga larger number of images even when a frequency range used for datatransmitted by the transmitter 11 or a communication band used by thenetwork interface 14 is limited.

In the receiving device 101 that receives the data on images from thetransmitting device 100, the image resolution increasing unit 6increases resolution in the direction of a motion of a subject based onthe difference between the positions of the subject included in theimages and alias components included in the data. The increase in theresolution makes it possible to reduce the number of alias components inthe direction of the motion of the subject.

Since an appropriate one or more of the low pass filters 21 is used tolimit the frequency range of a frequency component (of the image signal)in a direction in which no subject moves, the frequency range of theimage signal output from the transmitting device 100 is limited.

The image resolution increasing unit 6 is therefore capable of reducingalias components in the direction in which a subject moves and aliascomponents in a direction in which no subject moves. In addition, theimage resolution increasing unit 6 is capable of generating an imagehaving higher resolution than that of the images included in the imagesignal output from the transmitting device 100 and outputting thegenerated image.

Briefly, in the image transmission/reception system according to thefirst embodiment, which has the transmitting device 100 and thereceiving device 101, the transmitting device 100 reduces the size ofthe image(s) to reduce the amount of data to be transmitted, and thereceiving device 101 decodes the image(s) and generates an image havinghigher resolution. The receiving device 101 then restores the image. Inthis case, the receiving device 101 is capable of reducing aliascomponents included in the image obtained after the resolution isincreased, and preventing degradation in the image quality due to noiseor the like.

The pixels representing the subject moving only in the horizontaldirection are described above with reference to FIG. 2. Pixelsrepresenting a subject moving in the vertical direction or in adirection oblique to the vertical and horizontal directions, however,may be applied to the first embodiment.

The selector 22 may perform low pass filtering on the image signal tocut high frequency component of the image signal. The selector 22 maynot perform low pass filtering on a frequency component in the directionof the motion of the subject and may perform low pass filtering on theimage signal to cut high frequency component (of the image signal) in adirection perpendicular to the direction of the motion of the subject.

The above processing performed by the selector 22 makes it possible toadjust the amount of alias components (or to cause a small amount ofalias components to remain) in a direction in which no subject moves.

The spatial low pass filters 21 according to the first embodiment may bedifferent in type from each other depending on the number of spatialdirections of an image, as shown in FIG. 2. The low pass filters 21outputs respective signals, and the selector 22 selects one of thesignals output from the low pass filters 21 based on the directions ofthe motion vectors included in the motion vector information output fromthe motion detector 8. This makes it possible to reduce alias componentsincluded in the image obtained after the image resolution increasingunit 6 increases the resolution.

When the images do not include any motion (the motion vectors are equalto zero), or when the amount of a motion of a subject included in theimages is on the basis of integer pixels, the image resolutionincreasing unit 6 cannot increase the resolution. In this case, theselector 22 may select a signal output from a spatially invariant lowpass filter that performs low pass filtering regardless of the directionof a motion of a subject. In addition, the spatially invariant low passfilter may be used even when the values of the motion vectors are notequal to zero or are not equal to the amount of an integer pixel(s) (oreven when the values of the motion vectors are close to zero or arenearly equal to the amount of an integer pixel(s)).

The image reduction unit 2 according to the first embodiment may have aconfiguration shown in FIG. 4. The image reduction unit 2 shown in FIG.4 includes a coefficient generator 41, a spatial low pass filter 42, anda pixel subsampling unit 43. The coefficient generator 41 receives themotion vector information from the motion detector 8 and generates acoefficient(s) based on the received motion vector information. Aspatial low pass filter 42 shown in FIG. 4 is different from the lowpass filters 21 shown in FIG. 2. The spatial low pass filter 42 switchesa set coefficient(s) to the coefficients generated by the coefficientgenerator 41. The coefficients correspond to the low pass filters 1, 2and the like (low pass filters 21) shown in FIG. 1. The pixelsubsampling unit 43 subsamples pixels in order to reduce the size ofeach image.

In the abovementioned way, the coefficient generator 41, the spatial lowpass filter 42, and the pixel subsampling unit 43 in FIG. 4 can achieveoperations similar to those performed by the spatial low pass filters 21and the selector 22 shown in FIG. 2. In the image reduction unit 2having the configuration shown in FIG. 4, a plurality of spatial lowpass filters is not required. The image reduction unit 2 shown in FIG. 4can be achieved with a simpler hardware configuration.

Next, a description will be made of the processing for increasingresolution by the image resolution increasing unit 6 included in thereceiving device 101 according to the first embodiment. In the followingdescription, it is assumed that a plurality of images input to the imageresolution increasing unit 6 according to the first embodiment is aplurality of frames (image frames).

First, the image resolution increasing unit 6 performs, for example,three types of processing, (1) position estimation, (2) wide frequencyrange interpolation, and (3) weighted sum, to perform processing forincreasing resolution. The position estimation (1) is to use image datapieces included in the respective input image frames and estimate adifference between sampling phases of the image data pieces (positionsof sampled image data pieces). The wide frequency range interpolation(2) is to use a wide band low pass filter for passing the image datapieces, alias components, and all high frequency components included inthe original image signal in order to interpolate pixels (samplingpoints) and increase the pixel densities of the image data pieces. Theweighed sum (3) is to use weighting coefficients obtained based on thesampling phases of the high-density data pieces in order to cancel andremove alias components generated due to the pixel sampling, and restorehigh frequency components of the original image signal.

FIGS. 12A to 12E show an outline of the technique for increasingresolution. In FIG. 12A, it is assumed that the image resolutionincreasing unit 6 receives a frame #1(1201), a frame #2(1202), and aframe #3(1203), which are present at different times from each other onthe time axis, and combines the frames #1 to #3 to obtain a frame (1206)to be output. For the sake of simplicity, it is assumed that a motion(1204) of a subject occurs in the horizontal direction andone-dimensional signal processing is performed on data on a horizontalline (1205) shown in FIG. 12A to perform the processing for increasingresolution. In this case, the position of a signal waveform of the frame#1(1201) is different from the position of a signal waveform of theframe #2(1202) depending on the amount of the motion (1204) of thesubject, as shown in FIGS. 12B and 12D. The difference of the positionsis calculated by the position estimation (1) described above. In orderto eliminate the difference between the positions of the signalwaveforms, motion compensation (1207) is performed on the frame#2(1202),as shown in FIG. 12C. The phase difference θ (1211) between samplingphases (1209) and (1210) of pixels (1208) of the frames is calculated.The wide frequency range interpolation (2) and the weighted sum (3) areperformed based on the phase difference θ (1211). As shown in FIG. 12E,each of new pixels (1212) is created at a central position phasedifference (θ=π) between original pixels (1208) to increase theresolution. The weighted sum (3) will be described later. Although thesubject may not only move in the horizontal direction, but also rotate,be enlarged, or be reduced, a minute motion or slow motion of thesubject can be approximated to a locally horizontal motion.

In this case, the image resolution increasing unit 6 has a firstconfiguration that is the same as one of configurations (for performinghigh resolution processing) described in JP-A-8-336046, JP-A-9-69755,and Non-Patent Document 1. To perform the weighted sum (3) in the casewhere the image resolution increasing unit 6 has the firstconfiguration, the image resolution increasing unit 6 can use at leastthree signals of image frames as shown in FIG. 13. In this case, it ispossible to obtain resolution larger by twice than that of the images ina one-dimensional direction.

Next, a description will be made of the processing for increasingresolution by the image resolution increasing unit 6 having the firstconfiguration with reference to FIGS. 13A to 13C. FIGS. 13A to 13C showfrequency spectrums of frequency components in a one-dimensionalfrequency domain. In FIGS. 13A to 13C, the distance from a frequencyaxis indicates a signal intensity, and a rotation angle centering on thefrequency axis indicates a phase. The weighted sum (3) will be describedbelow.

In the wide frequency range interpolation (2), wide range low passfiltering is performed to pass a frequency component within a frequencyrange (from a frequency of 0 Hz to the sampling frequency fs) larger bytwice than that from a frequency of 0 Hz to the Nyquist frequency (halfof the sampling frequency) in order to interpolate the pixels. After thepixel interpolation, the sum of the components (hereinafter called asoriginal components) that are the same as those of the original imagesignal and alias components generated depending on the sampling phasescan be obtained. It is well known that when the wide frequency rangeinterpolation (2) is performed on the signals of the three frames in theabove case, the phases of the original components (1301), (1302), and(1303) of the frames are identical to each other as shown in FIG. 13A,and the phases of the alias components (1304), (1305), and (1306) areshifted depending on differences between the sampling phases of theframes. To easily understand the relationship of the phases, therelationship of the phases of the original components of the frames isshown in FIG. 13B, and the relationship of the phases of the aliascomponents of the frames is shown in FIG. 13C.

Coefficients to be multiplied by the signals of the three frames areappropriately selected to perform the weighted sum (3). Therefore, thealias components (1304), (1305), and (1306) of the frames can becancelled out each other and removed. Only the original components canbe extracted. In this case, the vector sum of the alias components(1304), (1305), and (1306) of the frames is equal to zero. That is, inorder to set both components in the direction of a Re axis (real axis)and components in the direction of an Im axis (imaginary axis) to bezero, at least three alias components are required. When at least threesignals of frames are used, the resolution can be increased by twice,i.e., a single alias component can be removed.

FIG. 11 shows a second configuration of the image resolution increasingunit 6. The image resolution increasing unit 6 having the secondconfiguration is capable of increasing resolution in a one-dimensionaldirection by twice when at least two signals of frames are used.

The image resolution increasing unit 6 includes an input section 1100, aposition estimation unit 1101, a motion compensation and rate increasingunit 1115, a phase shift unit 1116, an alias component removing unit1117, and an output section 1118. The input section 1100 receives aplurality of images included in an image signal from the image memory 9shown in FIG. 1. The position estimation unit 1101 uses a sampling phase(sampling position) of a pixel (to be processed) on the frame #1 inputto the input section 1100 as a standard to estimate the position of apixel (corresponding to the pixel on the frame #1) on the frame #2 andobtain a difference (sampling phase difference) θ 1102 between thesampling phase of the pixel on the frame #1 and the position of thepixel on the frame #2 input to the input section 1100. The samplingphase difference θ 1102 is equivalent to the motion vector information.The sampling phase difference θ 1102 can be calculated by motiondetection processing.

The motion compensation and rate increasing unit 1115 has rateincreasing units 1103 and 1104. The rate increasing units 1103 and 1104perform motion compensation on the frame #2 by using the sampling phasedifference θ 1102 to position the frame #2 with respect to the frame #1.The rate increasing units 1103 and 1104 increase the number of pixels ofthe frame #1 and the number of pixels of the frame #2 by twice toincrease the pixel densities of image data pieces of the frames #1 and#2, respectively. The phase shift unit 1116 shifts the phases of theimage data pieces having the increased pixel densities by respectiveconstant quantities. The phase shift unit 1116 includes π/2 phaseshifters 1106 and 1108, which are capable of shifting the phases of theimage data pieces by respective constant quantities. The phase shiftunit 1116 also includes delay compensators 1105 and 1107, which arecapable of delaying the signals of the frames #1 and #2 having theincreased pixel densities to compensate phase delays generated by theπ/2 phase shifters 1106 and 1108. The alias component removing unit 1117includes a coefficient determination unit 1109, multipliers 1110, 1111,1112, and 1113, and an adder 1114. The coefficient determination unit1109 generates coefficients C0, C1, C2, and C3 based on the samplingphase difference θ 1102. The multipliers 1110 multiplies a signal outputfrom the delay compensator 1105 by the coefficient C0 generated by thecoefficient determination unit 1109 to obtain the product. Themultipliers 1111 multiplies a signal output from the π/2 phase shifter1106 by the coefficient C1 generated by the coefficient determinationunit 1109 to obtain the product. The multipliers 1112 multiplies asignal output from the delay compensator 1107 by the coefficient C2generated by the coefficient determination unit 1109 to obtain theproduct. The multipliers 1113 multiplies a signal output from the π/2phase shifter 1108 by the coefficient C3 generated by the coefficientdetermination unit 1109 to obtain the product. The adder 1114 calculatesthe sum of the products obtained by multipliers 1110 to 1113 to generatea signal (indicating the sum) to be output. The signal is output fromthe adder 1114 to the output section 1118. The output section 1118 thenoutputs the signal.

The position estimation 1101 may be configured by using a conventionaltechnique. Details of the rate increasing units 1103 and 1104, the π/2phase shifters 1106 and 1108, and the alias component removing unit 1117will be described later.

FIGS. 14A to 14C show operations of the image resolution increasing unit6 (shown in FIG. 11) having the second configuration. FIGS. 14A to 14Cshow the signals output from the delay compensators 1105 and 1107, andthe signals output from the π/2 phase shifters 1106 and 1108 in aone-dimensional frequency domain. In FIG. 14A, the signal (of the frame#1) having the increased pixel density and output from the delaycompensator 1105 indicates the sum of the original component 1401 and analias component 1405 generated based on the original sampling frequency(fs). Also, in FIG. 14A, the signal (of the frame #2) having theincreased pixel density and output from the delay compensator 1107indicates the sum of the original component 1402 and an alias component1406 generated based on the original sampling frequency (fs). In thiscase, the phase of the alias component 1406 is shifted by the samplingphase difference θ 1102. The signal (of the frame #1) having theincreased pixel density and output from the π/2 phase shifter 1106indicates the sum of the original component 1403 obtained after thephase of the original component 1403 is shifted by π/2 and an aliascomponent 1407 obtained after the phase of the alias component 1407 isshifted by π/2. The signal (of the frame #2) having the increased pixeldensity and output from the π/2 phase shifter 1108 indicates the sum ofthe original component 1404 obtained after the phase of the originalcomponent 1404 is shifted by π/2 and an alias component 1408 obtainedafter the phase of the alias component 1408 is shifted by π/2. FIGS. 14Band 14C respectively show the original components and the aliascomponents, which are extracted from the drawing of FIG. 14A, in orderto simplify the relationship of the phases of the components shown inFIG. 14A. It is possible to cancel out the alias components each otherand only extract the original components by performing the following.That is, components in the Re axial direction are defined as 1, andcomponents in the Im axial direction are defined as 0, in order tocalculate the vector sum of the four original components shown in FIG.14B, while components in the Re axial direction are defined as 0, andcomponents in the Im axial direction are defined as 0, in order tocalculate the vector sum of the four alias components shown in FIG. 14C.Then, a coefficient to be multiplied by each of the components isdetermined, and the weighted sum is performed on the components by usingthe determined coefficients. Therefore, an image signal processingdevice capable of doubling image resolution in a one-dimensionaldirection can be achieved by using only two frames (frame images). Themethod (described above) for determining a coefficient will be describedin detail later.

FIG. 15 shows operations of the rate increasing units 1103 and 1104included in the image resolution increasing unit 6 (shown in FIG. 11)having the second configuration. In FIG. 15, a frequency is plottedalong the abscissa axis, and a gain (ratio of the amplitude of an outputsignal to the amplitude of an input signal) is plotted along theordinate axis. FIG. 15 shows frequency versus gain characteristics ofthe rate increasing units 1103 and 1104. Each of the rate increasingunits 1103 and 1104 doubles the number of pixels to increase the pixeldensity of image data by using a frequency (2fs) larger by twice thanthe sampling frequency (fs) of the original signal as a new samplingfrequency and placing sampling points (=zero points) at respectivecentral positions between the original pixels. In addition, each of therate increasing units 1103 and 1104 perform filtering to pass afrequency component ranging from a frequency of −fs to a frequency of+fs in order to obtain the gain of 2.0. As shown in FIG. 15, the aboveoperations are repeated for each frequency range of 2fs by integralmultiple due to symmetry of the digital signal.

FIG. 16 shows a detail example of the rate increasing units 1103 and1104 included in the image resolution increasing unit 6 having thesecond configuration shown in FIG. 11. FIG. 16 shows filer tapcoefficients obtained by performing inverse Fourier transform onfrequency components having characteristics shown in FIG. 15. The tapcoefficients Ck (k is an integer) form a sinc function that is generallyknown. The sinc function is shifted by a value of −θ to compensate thesampling phase difference θ 1102. That is, Ck=2 sin(πk+θ)/(πk+θ). In therate increasing unit 1103, the sampling phase difference θ 1102 is equalto zero, and Ck=2 sin(πk)/(πk). In addition, the sampling phasedifference θ 1102 is expressed by a phase difference on an integer pixelbasis (2π) and a phase difference on a sub-pixel basis. The sub-pixel issmaller than a single pixel. The phase difference on an integer pixelbasis can be compensated by simple pixel shifting. The phase differenceon a sub-pixel basis may be compensated by using filters of the rateincreasing units 1103 and 1104.

FIGS. 17A and 17B show operations of the π/2 phase shifter 1106 and 1108included in the image resolution increasing unit 6 having the secondconfiguration shown in FIG. 11. Generally known Hilbert transformers maybe used as the π/2 phase shifter 1106 and 1108. In FIG. 17A, a frequencyis plotted along the abscissa axis and a gain (ratio of the amplitude ofan input signal to the amplitude of an output signal) is plotted alongthe ordinate axis. FIG. 17A shows frequency versus gain characteristicsof the Hilbert transformers. Each of the Hilbert transformers uses afrequency (2fs) larger by twice than the sampling frequency (fs) of theoriginal signal as a new sampling frequency and passes a frequencycomponent ranging from a frequency of −fs to a frequency of +fs otherthan a frequency of 0 Hz in order to obtain the gain of 1.0. In FIG.17B, a frequency is plotted along the abscissa axis, and a phasedifference (between the phase of the input signal and the phase of theoutput signal) is plotted along the ordinate axis. FIG. 17B showsfrequency versus phase difference characteristics of the Hilberttransformers. The phases of the frequency components within a frequencyrange from 0 to fs are delayed by π/2. The phases of the frequencycomponents within a frequency range from 0 to −fs are forwarded by π/2.As shown in FIGS. 17A and 17B, the above operations are repeated foreach frequency range of 2fs by integral multiple due to symmetry of thedigital signal.

FIG. 18 shows the case where the Hilbert transformers are used as theπ/2 phase shifters 1106 and 1108 included in the image resolutionincreasing unit 6 having the second configuration shown in FIG. 11. FIG.18 shows filter tap coefficients obtained by performing inverse Fouriertransformation on frequency components having characteristics shown inFIGS. 17A and 17B. In this case, each tap coefficient Ck is equal tozero when k=2m (m is an integer), and each tap coefficient Ck is −2/(πk)when k=2m+1.

Differentiators may be used as the π/2 phase shifters 1106 and 1108 usedin the present embodiment. In this case, when each of thedifferentiators differentiates a general expression of cos(ωt+α), whichshows a sine-wave, with respect to a time t and multiplies thedifferentiated expression by a value of 1/ω, the following expression isestablished: d(cos(ω+α))/dt*(1/ω)=−sin(ωt+α)=cos(ωt+α+π/2). Therefore,the differentiators can achieve the functions of the π/2 phase shifters.Each of the differentiators may obtain the difference between a value ofa target pixel and a value of a pixel adjacent to the target pixel, anduses a filter having frequency versus amplitude characteristics for thevalue of 1/ω to achieve the function of each of the π/2 phase shifters.

FIGS. 19A to 19D show a detail example of operations of the coefficientdetermination unit 109 included in the image resolution increasing unit6 having the second configuration shown in FIG. 11. As described above,the components in the Re axial direction are defined as 1, andcomponents in the Im axial direction are defined as 0, in order tocalculate the vector sum of the four original components shown in FIG.14B, while the components in the Re axial direction are defined as 0,and components in the Im axial direction are defined as 0, in order tocalculate the vector sum of the four alias components shown in FIG. 14C.As shown in FIG. 19A, an image signal processing device capable ofdoubling resolution in a one-dimensional direction can be achieved byusing only two frames (frame images) when coefficients to be multipliedby each of the components are determined. Simultaneous equations shownin FIG. 19B can be obtained based on the relationship of the phases ofthe components shown in FIGS. 14B and 14C when requirements shown inFIG. 19A are satisfied in the case where: C0 is a coefficient for asignal (indicating the sum of the original component and alias componentof the frame #1 having the pixel density increased by the rateincreasing unit 1103) output from the delay compensator 1105 shown inFIG. 11; C1 is a coefficient for a signal (indicating the sum of the π/2phase-shifted original component and π/2 phase-shifted alias componentof the frame #1 having the pixel density increased by the rateincreasing unit 1103) output from the π/2 phase shifter 1106 shown inFIG. 11; C2 is a coefficient for a signal (indicating the sum of theoriginal component and alias component of the frame #2 having the pixeldensity increased by the rate increasing unit 1104) output from thedelay compensator 1107 shown in FIG. 11; and C3 is a coefficient for asignal (indicating the sum of the π/2 phase-shifted original componentand π/2 phase-shifted alias component of the frame #2 having the pixeldensity increased by the rate increasing unit 1104) output from the π/2phase shifter 1108 shown in FIG. 11. The simultaneous equations resultsin the relationships shown in FIG. 19C. The coefficient determinationunit 1109 may output the coefficients C0, C1, C2, and C3 obtained in theabovementioned way. As an example, FIG. 19D shows the coefficients C0,C1, C2, and C3 obtained when the sampling phase difference θ 1102 isshifted from 0 to 2π at intervals of π/8. This corresponds to the casewhere the position of the signal of the original frame #2 is estimatedwith 1/16 pixels of accuracy and a motion in the frame #2 with respectto the frame #1 is compensated.

Although each of the rate increasing units 1103 and 1104 and the π/2phase shifters 1106 and 1108 requires an infinite number of taps inorder to obtain ideal characteristics, no practical problem arises evenwhen the number of taps is limited for simplicity. In the case where thenumber of the taps is limited, a general window function (e.g., Hanningwindow function or Hamming window function) may be used. When acoefficient of each tap of the simplified Hilbert transformer is set toa value bilaterally symmetric with respect to C0 (i.e., C(−k)=−Ck (k isan integer)), the phase can be shifted by a constant quantity.

As described above, the image resolution increasing unit 6 (shown inFIG. 1) having the configuration shown in FIGS. 11 to 19D is capable ofgenerating an image having higher resolution from a plurality of images.

Especially, the image resolution increasing unit 6 having the secondconfiguration shown in FIG. 11 is capable of generating a single imagehaving higher resolution from two images.

The image resolution increasing unit 6 may use the motion vectorsincluded in the stream decoded by the decoder 5 to perform theprocessing for increasing resolution, as shown in FIG. 1. In this case,the motion detection processing to be performed by the positionestimation unit 1101 included in the image resolution increasing unit 6shown in FIG. 11 may not be required. This reduces the amount of data tobe processed by the receiving device 101.

In the image transmission/reception system or the method fortransmitting and receiving an image according to the first embodimentdescribed above, the transmitting device 100 reduces the size of theimage to reduce data to be transmitted, and the receiving device 101increases the resolution and restores the image. In this case, thetransmitting device 100 selectively performs low pass filtering based onthe direction of a motion included in the image signal and generates animage signal including alias components that vary depending on thedirection of the motion, and the receiving device 101 can increase theresolution and reduce the alias components.

In other words, the transmitting device 100 is capable of transmittingthe image signal with a small amount of data, and receiving device 101is capable of restoring the image having a higher image quality.

The transmitting device 100 included in the image transmission/receptionsystem according to the first embodiment described above reduces thesize of the image(s) while selectively performing low pass filteringbased on the direction of a motion included in an image signal, andgenerates an image signal suitable for the receiving device 101 thatgenerates a high-resolution image with minimized degradation in theimage quality (the degradation in the image quality is caused by due tonoise or the like). In addition, the transmitting device 100 is capableof transmitting image data encoded at a high compression rate.

The receiving device 100 included in the image transmission/receptionsystem according to the first embodiment described above receives theimage signal subjected to the low pass filtering for reducing aliascomponents in a direction other than the direction in which a subjectmoves and alias components in the direction of the motion (the reductionin the alias components in the direction other than the direction of themotion is larger than the reduction in the alias components in thedirection of the motion), reduces the alias components in the directionof the motion of the subject, increases the resolution, and generates ahigh-resolution image with minimized degradation in the image quality(the degradation is caused by due to noise or the like).

Second Embodiment

FIG. 6 shows an example of an image storage device 600 according to asecond embodiment of the present invention. The receiver 12, the networkinterface 14, the decoder 5, the image resolution increasing unit 6, theimage memory 9, the display unit 10, and the image signal output section7, which are included in the image storage device 600, are the same asthose provided in the receiving device 101 of the imagetransmission/reception system according to the first embodiment.

In addition, the image signal input section 1, the image reduction unit2, the encoder 3, the motion detector 8, and the motion vectortransformer 15, which are included in the image storage device 600, arethe same as those provided in the transmitting device 100 of the imagetransmission/reception system according to the first embodiment.

The image storage device 600 receives an image signal from anotherexternal device by means of the image signal input section 1. The imagestorage device 600 may receive an image signal from another externaldevice by means of any of the receiver 12, the network interface 14, andan imaging unit 61 such as a camera unit. For example, the image storagedevice 600 receives a television broadcast wave including an imagesignal by means of the receiver 12. In addition, the image storagedevice 600 receives an image signal from a server or the like through anetwork by means of the network interface 14. Furthermore, the imagingunit 61 images a subject by using a lens, an optical sensor or the liketo generate an image signal.

Those image signals include an encoded stream and a non-encoded stream.The image storage device 600 has a switch 62 that receives signals fromthe receiver 12, the network interface 14, the image signal inputsection 1, and the imaging unit 61. When the switch 62 receives anencoded signal, the switch 62 outputs the encoded signal to a decoder63. The decoder 63 decodes the encoded signal and outputs the decodedsignal to the image reduction unit 2. On the other hand, when the switch62 receives a non-encoded signal, the switch 62 outputs the non-encodedsignal to the image reduction unit 2.

The motion detector 8 and the image reduction unit 2 perform operationsin the same manner as those of the motion detector 8 and the imagereduction unit 2 which are included in the transmitting device 100 ofthe image transmission/reception system according to the firstembodiment. The image reduction unit 2 selectively performs low passfiltering based on the direction(s) of the motion vectors calculated bythe motion detector 8 and reduces the size of each image included in theimage signal.

In the image storage device 600 having another configuration, the imagereduction unit 2 shown in FIG. 6 uses the motion vectors included in theencoded stream decoded by the decoder 63 to select the direction of afrequency component to be subjected to low pass filtering based on thedirection(s) of the motion vectors. In this case, the image reductionunit 2 is capable of performing the low pass filtering based on thedirection(s) of the motion vectors without the motion detector 8.

The encoder 3 shown in FIG. 6 encodes a signal. The encoder 3 mayperform motion detection processing and use motion vectors to encode animage.

In the image storage device 600 having another configuration, theencoder 3 may use information on the motion vectors calculated by themotion detector 8. In this case, the size of each image included in theimage signal used for the calculation of the motion vectors by themotion detector 8 is different from the size of an image included in theimage signal encoded by the encoder 3. In such a manner as described inthe first embodiment, the motion vector transformer 15 changes thenumber of the motion vectors (generated by the motion detector 8) andthe length of each of the motion vectors based on the image reductionrate of the image reduction unit 2 to generate motion vectors that canbe used by the encoder 3. When the encoder 3 uses the motion vectorsgenerated by the motion vector transformer 15, motion detectionprocessing to be performed by the encoder 3 is not necessary. Thisresults in a reduction in the amount of data to be processed.

In the image storage device 600 having another configuration, the motionvector transformer 15 may change the motion vectors included in thestream decoded by the decoder 63 to generate motion vectors that can beused for encoding by the encoder 3.

A storage unit 64 included in the image storage device 600 stores theimage signal encoded by the encoder 3.

As described above, the image storage device 600 according to the secondembodiment reduces the size of an image included in an image signal andthen stores the signal. The image storage device 600 is thereforecapable of storing the image signal compressed at a higher compressionrate than that in the case where the image signal is encoded and storedwithout a reduction in the size of the image.

Next, the decoder 5 shown in FIG. 6 decodes the encoded image signalstored in the storage unit 64. The decoded image signal is stored in theimage memory 9. The image resolution increasing unit 6 performs the sameprocessing as that in the first embodiment on a plurality of imagesincluded in the image signal stored in the image memory 9 to generate animage having high resolution. The processing performed by the imageresolution increasing unit 6 is described in the first embodiment and adetail description thereof is omitted.

Lastly, the display unit 10 displays the generated high-resolutionimage. In addition, the image signal output section 7 outputs thegenerated high-resolution image to another external device.

The image resolution increasing unit 6 may use the motion vectorsincluded in the encoded stream decoded by the decoder 5 to perform theprocessing for increasing resolution. In this case, the motion detectionprocessing to be performed by the position estimation unit 1101 includedin the image resolution increasing unit 6 shown in FIG. 11 may not berequired in the second embodiment, in such a way as described in thefirst embodiment. This reduces the amount of data to be processed by theimage storage device 600.

In the image storage device 600 according to the second embodimentdescribed above, the image reduction unit 2 uses the motion vectorsincluded in the stream decoded by the decoder 63 to select the directionof a frequency component to be subjected to low pass filtering based onthe direction(s) of the motion vectors. The motion vector transformer 15may use the motion vectors included in the stream decoded by the decoder63 to generate motion vectors that can be used for encoding by theencoder 3. The image resolution increasing unit 6 may use the motionvectors included in the stream decoded by the decoder 5 to perform theprocessing for increasing the resolution. In this case, the imagestorage device 600 can perform the abovementioned operations without themotion detection processing. Therefore, a hardware device for performingthe motion detection processing is not required for the image storagedevice 600. It is therefore possible to realize, at a low cost, theimage storage device 600 capable of storing an image signal compressedat a higher compression rate and generating a high-resolution image withminimized degradation in the image quality (the degradation in the imagequality is caused by due to noise or the like).

As described above, the image storage device 600 according to the secondembodiment reduces the size of an image included in a received imagesignal, performs low pass filtering to reduce alias components in adirection other than the direction of a motion included in the image andalias components in the direction of the motion, encodes the imagesignal to compress the image signal and reduce the amount of data of theimage signal, and stores the encoded image signal in the storage unit.In this case, the amount of the reduced alias components in thedirection other than the direction of the motion is larger than that ofthe reduced alias components in the direction of the motion. Inaddition, the image storage device 600 is capable of generating ahigh-resolution image with minimized degradation in the image quality bydecoding the image signal stored in the storage unit and increasing theresolution with a reduction in the alias components in the direction ofthe motion.

Furthermore, the image storage device 600 is capable of displaying oroutputting the high-resolution image with minimized degradation in theimage quality (the degradation in the image quality is caused by due tonoise or the like) after encoding the image signal at a high rate andstoring the encoded image signal.

Third Embodiment

FIG. 7 shows an example of an image transmission/reception systemaccording to a third embodiment of the present invention. The imagetransmission/reception system according to the third embodiment includesa transmitting device 700 and a receiving device 701. The imagetransmission/reception system according to the third embodiment isdifferent from the image transmission/reception system according to thefirst embodiment. In the image transmission/reception system accordingto the third embodiment, the low pass filtering that is selectivelyperformed based on the direction of a motion included in an image is notperformed in the transmitting device 700. The transmitting device 700reduces the sizes of the images and transmits an image signal includingthe reduced images. The receiving device 701 performs processing forincreasing resolution of the reduced images transmitted by thetransmitting device 700 and then performs low pass filtering on theimage signal to pass a low frequency component in a direction other thanthe direction of a motion included in the images. The receiving device701 is therefore capable of generating a high-resolution image withminimized degradation in the image quality (the degradation in the imagequality is caused by due to noise or the like).

The transmitting device 700 according to the third embodiment has animage reduction unit 71 that is different from the image reduction unit2 included in the transmitting device 100 according to the firstembodiment. That is, the image reduction unit 71 does not perform thelow pass filtering based on the direction of a motion included in imagesand generates a reduced image having an alias component.

Other parts of the transmitting device 700 according to the thirdembodiment are the same as those of the transmitting device 100according to the first embodiment, and description thereof is omitted.

In such a manner as described in the first embodiment, the imageresolution increasing unit 6 may use the motion vectors included in theencoded stream decoded by the decoder 5 to perform the processing forincreasing resolution. In this case, the motion detection processing tobe performed by the position estimation unit 1101 included in the imageresolution increasing unit 6 shown in FIG. 11 may not be required. Thisreduces the amount of data to be processed by the receiving device 701.

The receiving device 701 according to the third embodiment has a motionvector acquirer 72 and an image corrector 73, which are not included inthe receiving device 101 according to the first embodiment.

The motion vector acquirer 72 acquires information (motion vectorinformation) on motion vectors included in the image signal decoded bythe decoder 5. The motion vector information is included in an encodedstream generated by the encoder 3 included in the transmitting device700.

In addition, the motion vector acquirer 72 acquires motion vectorinformation (sampling phase difference θ 1102) calculated by theposition estimation unit 1101 included in the image resolutionincreasing unit 6.

The motion vector information acquired by the motion vector acquirer 72is obtained by a calculation of the sizes of images obtained before theimage resolution increasing unit 6 performs the processing forincreasing resolution. The motion vector acquirer 72 therefore performsprocessing for changing the sizes of the motion vectors based on therate of enlarging the size of the image in the processing (forincreasing resolution) performed by the image resolution increasing unit6 to ensure that the motion vector information acquired by the motionvector acquirer 72 can be used for the size of the image obtained afterthe image resolution increasing unit 6 performs the processing. Themotion vector acquirer 72 outputs, to the image corrector 73,information on the motion vectors subjected to the processing (forchanging the sizes of the motion vectors).

Since the motion vector acquirer 72 acquires the motion vectorinformation and performs the processing for changing the sizes of themotion vectors, the image corrector 73 uses the motion vectors andreduces the amount of data to be processed. This results from the factthat the image corrector 73 does not need to perform motion detectionprocessing.

The image corrector 73 performs processing for correcting thehigh-resolution image generated by the image resolution increasing unit6. Details of the processing will be described below.

The transmitting device 700 does not perform low pass filtering on animage signal (to be transmitted to the receiving device 701) based onthe direction of a motion included in an image. Therefore, an imagesignal input to the image resolution increasing unit 6 includes aliascomponents in directions in the image.

The image resolution increasing unit 6 reduces the alias components inthe direction of a motion included in the image signal and increasesresolution in the direction of the motion. Although alias components inthe direction of the motion included in the high-resolution imagegenerated by the image resolution increasing unit 6 are reduced, aliascomponents in a direction other than the direction of the motionincluded in the high-resolution image are not reduced and remain.

The image corrector 73 performs low pass filtering on thehigh-resolution image generated by the image resolution increasing unit6 to reduce alias components. However, if the image corrector 73performs the low pass filtering on an image signal including the imageto pass a low frequency component in the direction of a motion includedin the image, a high frequency component obtained after the imageresolution increasing unit 6 increases the resolution is removed. Thisreduces the effect of the processing for increasing the resolution bymeans of the image resolution increasing unit 6.

To prevent the reduction in the effect, the image corrector 73 accordingto the third embodiment uses the motion vectors acquired by the motionvector acquirer 72 to perform appropriate low pass filtering to pass alow frequency component in a direction other than the direction of themotion included in the image. The image corrector 73 according to thethird embodiment is therefore capable of reducing the remaining aliascomponents and generating a high-resolution image with minimizeddegradation in the image quality (the degradation in the image qualityis caused by due to noise or the like) while maintaining the effect ofthe processing for increasing the resolution by means of the imageresolution increasing unit 6.

Next, a description will be made of details of the image corrector 73included in the receiving device 701 according to the third embodimentwith reference to FIG. 8.

The image signal received by the image corrector 73 is input in parallelto each of spatial low pass filters 81 having respective characteristicsdifferent from each other. Each of the spatial low pass filters 81outputs a signal to a selector 82. The selector 82 selects one of thesignals output from the low pass filters 81 and outputs the selectedsignal. The low pass filters 81 have respective differentcharacteristics of frequency ranges to be limited and directions similarto the low pass filters 21 in the first embodiment. In this case, theselector 82 uses information on the motion vectors input to the selector82 from the motion vector acquirer 72 to perform the low pass filteringin order to limit a frequency range of a frequency component in adirection other than the direction of a motion included in the images ofthe image signal based on the signals output from the low pass filters81.

The image signal received by the image corrector 73 includes aliascomponents in a direction other than the direction of the motionincluded in the images of the image signal. Since the selector 82performs the abovementioned processing, the receiving device 701according to the third embodiment is capable of reducing aliascomponents in the direction other than the direction of the motionincluded in the images of the image signal while maintaining a highfrequency component in the direction of the motion of the imagesubjected to the processing (for increasing resolution) performed by theimage resolution increasing unit 6. The receiving device 701 istherefore capable of generating a high-resolution image with minimizeddegradation in the image quality (the degradation in the image qualityis caused by due to noise or the like).

The image corrector 73 according to the third embodiment is differentfrom the image reduction unit 2 according to the first embodiment anddoes not perform the processing for reducing the size of an image. Theselector 82 included in the image corrector 73 is different from theselector 22 included in the image reduction unit 2 and does not performthe processing for subsampling pixels.

The image corrector 73 according to the third embodiment may have aconfiguration shown in FIG. 9. The image corrector 73 having theconfiguration shown in FIG. 9 includes a coefficient generator 91 and aspatial low pass filter 92. The coefficient generator 91 performs thesame processing as that performed by the coefficient generator 41 (shownin FIG. 4) according to the first embodiment. The spatial low passfilter 92 performs the same processing as that performed by the spatiallow pass filter 42 (shown in FIG. 4) according to the first embodiment.Specifically, the coefficient generator 91 generates coefficients forthe spatial low pass filter 92 based on the motion vector informationreceived by the coefficient generator 91 from the motion vector acquirer72. The coefficients correspond to those of the low pass filter 1, thelow pass filter 2 and the like (which are the low pass filters 81) shownin FIG. 8. The spatial low pass filter 92 switches a set coefficient(s)to the coefficients generated by the coefficient generator 91.

The coefficient generator 91 and the spatial low pass filter 92 shown inFIG. 9 can therefore perform operations similar to those of the spatiallow pass filters 81 and the selector 82 shown in FIG. 8. In the imagecorrector 73 having the configuration shown in FIG. 9, multiple types ofspatial low pass filters is not required. The image corrector 73 can beachieved with a simpler hardware configuration.

In the receiving device 701 included in the image transmission/receptionsystem according to the third embodiment described above, the imageresolution increasing unit 6 uses the motion vectors included in thestream decoded by the decoder 5 to perform the processing for increasingresolution, and the image corrector 73 uses motion vectors obtained bychanging the sizes of the motion vectors (included in the stream decodedby the decoder 5) by means of the motion vector acquirer 72 to selectthe direction of a frequency component to be subjected to the low passfiltering. In this case, the receiving device 701 shown in FIG. 7 canperform the abovementioned operations without the motion detectionprocessing. Therefore, a hardware device for performing the motiondetection processing is not required for the receiving device 701. It istherefore possible to realize, at a low cost, the receiving device 701capable of generating a high-resolution image with minimized degradationin the image quality (the degradation in the image quality is caused bydue to noise or the like).

In the image transmission/reception system or the method fortransmitting and receiving an image according to the third embodimentdescribed above, the transmitting device 700 reduces the size of eachimage to reduce data to be transmitted, and the receiving device 701increases the resolution and restores the image. In this case, thetransmitting device 700 is capable of outputting the image signalincluding alias components, and the receiving device 701 is capable ofperforming the processing for increasing the resolution by using thealias components. The receiving device 701 increases resolution in thedirection of the motion included in the images of the image signal byusing alias components and performs low pass filtering to reduce aliascomponents in a direction other than the direction of the motion. Thereceiving device 701 is therefore capable of generating ahigh-resolution image with minimized degradation in the image quality ofthe image (the degradation in the image quality is caused by due tonoise or the like).

That is, it is possible to transmit the image signal with a small amountof data from the transmitting device and restore an image having higherresolution by the receiving device.

The transmitting device 700 included in the image transmission/receptionsystem according to the third embodiment described above reduces thesize of each of the images included in the image signal to reduce theamount of data to be transmitted and outputs the image signal includingalias components. It is therefore possible to transmit the image signal(that can be processed to increase the resolution by using aliascomponents in the receiving device) by using a small amount of data.

The receiving device 701 included in the image transmission/receptionsystem according to the third embodiment described above receives theimage signal including the alias components and reduces the aliascomponents in the direction of a motion included in the images toincrease the resolution. In addition, the receiving device 701 performslow pass filtering on the image signal to pass a low frequency componentin a direction other than the direction of the motion to reduce aliascomponents. The receiving device 701 is therefore capable of generatinga high-resolution image with minimized degradation in the image quality(the degradation in the image quality is caused by due to noise or thelike).

Fourth Embodiment

FIG. 10 shows an example of an image storage device 1000 according to afourth embodiment of the present invention. The image storage device1000 is different from the image storage device 600 according to thesecond embodiment. The image storage device 1000 reduces the size ofeach of images without performing the selective low pass filtering basedon the direction of a motion included in the images before an imagesignal including the images is stored in a storage unit 64. The imagestorage device 1000 performs processing for increasing resolution of thereduced images and performs the low pass filtering to pass a lowfrequency component in a direction other than the direction of themotion. The image storage device 1000 is capable of generating ahigh-resolution image with minimized degradation in the image quality(the degradation in the image quality is caused by due to noise or thelike) in order to reproduce the image stored in storage unit 64.

The image signal input section 1, the receiver 12, the network interface14, the imaging unit 61, the switch 62, the decoder 63, the storage unit64, the encoder 3, the decoder 5, the image resolution increasing unit6, the image memory 9, the display unit 10, and the image signal outputsection 7, which are included in the image storage device 1000, are thesame as those included in the image storage unit 600 according to thesecond embodiment, and description thereof is omitted.

Next, a description will be made of differences between the imagestorage device 1000 according to the fourth embodiment and the imagestorage device 600 according to the second embodiment.

Operations of the image signal input section 1, the receiver 12, thenetwork interface 14, the imaging unit 61, the switch 62, and thedecoder 63, which are included in the image storage device 1000, are thesame as those in the image storage device 600 according to the secondembodiment.

The image reduction unit 71 included in the image storage device 1000 isthe same as that included in the transmitting device 700 according tothe third embodiment, and therefore different from the image reductionunit 2 included in the image storage device 600 according to the secondembodiment. The image reduction unit 71 does not perform low passfiltering based on a motion included in images and generates reducedimages including alias components.

The encoder 3 encodes an image signal including the reduced imagesgenerated by the image reduction unit 71 and outputs the encoded imagesignal to the storage unit 64. The storage unit 64 stores the encodedimage signal.

In the image storage device 1000 having another configuration, themotion vector transformer 15 changes the sizes of motion vectorsincluded in a stream decoded by the decoder 63, and the encoder 3 mayuse the motion vectors to encode the image signal.

As described above, the image storage device 1000 according to thefourth embodiment reduces the size of each image included in an imagesignal and then stores the image signal. The image storage device 1000is therefore capable of storing the image signal compressed at a highercompression rate than that in the case where the image signal is encodedand stored without a reduction in the size of the image.

The decoder 5 decodes the encoded image signal stored in the storageunit 64. The decoded image signal is stored in the image memory 9. Theimage resolution increasing unit 6 performs the processing, which is thesame as that in the first embodiment, on a plurality of images includedin the image signal stored in the image memory 9 to generate an imagehaving high resolution. The processing performed by the image resolutionincreasing unit 6 is described above in the first embodiment, anddescription thereof is omitted.

In such a manner as described in the first embodiment, the imageresolution increasing unit 6 may use the motion vectors included in theencoded stream decoded by the decoder 5 to perform the processing forincreasing the resolution. In this case, the motion detection processingto be performed by the position estimation unit 1101 included in theimage resolution increasing unit 6 shown in FIG. 11 is not required.This reduces the amount of data to be processed in the image storagedevice 1000.

The alias components remain in the image signal stored in the storageunit 64 included in the image storage device 1000 according to thefourth embodiment regardless of the direction of a motion. Although theimage resolution increasing unit 6 performs the processing forincreasing resolution in the direction of the motion and reducing aliascomponents in the direction of the motion, alias components in adirection other than the direction of the motion remain in the imagesignal.

In the image storage device 1000 according to the fourth embodiment, theimage corrector 73 performs the same processing as that performed by theimage corrector 73 included in the receiving device 701 according to thethird embodiment. That is, the image corrector 73 performs the low passfiltering to pass a low frequency component in a direction other thanthe direction of a motion included in the image signal to reduce aliascomponents. Details of the image corrector 73 are described above in thethird embodiment, and description of the image corrector 73 according tothe fourth embodiment is omitted.

The processing performed by the motion vector acquirer 72 included inthe image storage device 1000 is the same as that performed by themotion vector acquirer 72 included in the receiving device 701 accordingto the third embodiment, and description thereof is omitted.

Lastly, the display unit 10 displays the generated high-resolutionimage. Alternatively, the image signal output section 7 outputs thegenerated high-resolution image to another external device.

In the image storage device 1000 according to the fourth embodimentdescribed above, for example, the encoder 3 encodes an image signal byusing motion vectors obtained by transforming, by means of the motionvector transformer 15, motion vectors included in the stream decoded bythe decoder 63; the image resolution increasing unit 6 performs theprocessing for increasing resolution by using the motion vectorsincluded in the stream decoded by the decoder 5; the image corrector 73selects the direction of a frequency component to be subjected to lowpass filtering by using the motion vectors (included in the streamdecoded by decoder 5) obtained by the transforming by means of themotion vector transformer 15. In this case, the image storage device1000 shown in FIG. 10 can perform the abovementioned operations withoutthe motion detection processing. Therefore, a hardware device forperforming the motion detection processing is not required for the imagestorage device 1000. It is therefore possible to realize, at a low cost,the image storage device 1000 capable of storing an image signalcompressed at a higher compression rate and generating a high-resolutionimage with minimized degradation in the image quality (the degradationin the image quality is caused by due to noise or the like).

As described above, the image storage device 1000 according to thefourth embodiment reduces the size of each image included in a receivedimage signal and encodes the image signal including alias components.The encoded image signal having a small amount of data can be stored inthe storage unit. In addition, the image storage device 1000 is capableof decoding the image signal stored in the storage unit, and reducingalias components in the direction of a motion included in the imagesignal to increase resolution. Furthermore, the image storage device1000 is capable of performing low pass filtering to pass a low frequencycomponent in a direction other than the direction of the motion includedin the image signal to reduce alias components, and generating ahigh-resolution image with minimized degradation in the image quality(the degradation in the image quality is caused by due to noise or thelike).

In addition, the image storage device 1000 according to the fourthembodiment described above reduces the size of the image included in theimage signal, stores the image signal compressed at a higher compressionrate, and then displays or outputs the high-resolution image withminimized degradation in the image quality (the degradation in the imagequality is caused by due to noise or the like).

While we have shown and described several embodiments in accordance withour invention, it should be understood that the disclosed embodiment issusceptible to changes and modifications without departing from thescope of the invention. Therefore, we do not intend to be bound by thedetails shown and described herein but intend to cover all such changesand modifications as fall within the ambit of the appended claims.

1. A method for transmitting and receiving an image signal by means of atransmitting device for transmitting the image signal and a receivingdevice for receiving the image signal, comprising: an image reductionstep of reducing an image included in the image signal and generating animage signal including the reduced image by means of the transmittingdevice; a transmitting step of transmitting the image signal includingthe reduced image by means of the transmitting device; a receiving stepof receiving the image signal transmitted by the transmitting device bymeans of the receiving device; and an image resolution increasing stepof using an alias component and information on a motion that areincluded in the received image signal to convert the image signal intoan image having high resolution by means of the receiving device.
 2. Themethod according to claim 1, wherein in the image reduction step,processing for reducing an image and low pass filtering is performed onthe image signal to cut a high frequency component in the direction ofthe motion included in the image signal, wherein the cut-off frequencyof the low pass filter for the image signal in the direction other thanthe direction of the motion is lower than the cut-off frequency of thelow pass filter for the image signal in the direction of the motion. 3.The method according to claim 2, wherein in the image resolutionincreasing step, the receiving device performs processing for increasingthe resolution in the direction of the motion included in the imagesignal.
 4. The method according to claim 1, further comprising an imagecorrection step of performing low pass filtering by means of thereceiving device after the image resolution increasing step, wherein inthe image reduction step, the transmitting device reduces the imageincluded in the image signal while the alias component remains in theimage signal; in the image resolution increasing step, the receivingdevice performs processing for increasing the resolution in thedirection of the motion; and in the image correction step, the receivingdevice performs low pass filtering on the image signal to pass a lowfrequency component in a direction other than the direction of themotion included in the image signal.
 5. The method according to claim 4,further comprising: an encoding step of encoding the image signalgenerated in the image reduction step to generate an encoded streamafter the image reduction step by means of the transmitting device,wherein the transmitting device transmits the encoded stream generatedin the encoding step, and the receiving device receives the encodedstream in the receiving step; and a decoding step of decoding theencoded stream received in the receiving step after the receiving stepby means of the receiving device, wherein in the image correction step,the receiving device selects the direction of a frequency component tobe subjected to low pass filtering by using a motion vector included inthe encoded stream decoded in the decoding step and performs the lowpass filtering on the image signal to pass the low frequency componentin the selected direction.
 6. A receiving device comprising: a receiverwhich receives an encoded stream included in an image signal; a decoderwhich decodes the encoded stream received by the receiver; an imageresolution increasing unit which performs processing on the image signaldecoded by the decoder to increase resolution by using an aliascomponent and motion information that are included in the image signalin order to generate a high-resolution image; and an image correctorwhich performs low pass filtering on an image signal including thehigh-resolution image generated by the image resolution increasing unit.7. The receiving device according to claim 6, wherein the imageresolution increasing unit performs the processing for increasing theresolution in the direction of a motion included in the image signal;and the image corrector performs low pass filtering on the image signalto pass a low frequency component in a direction other than thedirection of the motion included in the image signal.
 8. The receivingdevice according to claim 6, wherein the image resolution increasingunit performs the processing for increasing the resolution in thedirection of a motion included in the image signal; and the imagecorrector performs low pass filtering on the image signal to cut a highfrequency component in the direction of the motion included in the imagesignal, wherein the cut-off frequency of the low pass filter for theimage signal in the direction other than the direction of the motion islower than the cut-off frequency of the low pass filter for the imagesignal in the direction of the motion.
 9. The receiving device accordingto claim 6, wherein the image corrector selects the direction of afrequency component to be subjected to the low pass filtering by usingthe motion information included in the encoded stream decoded by thedecoder.
 10. The receiving device according to claim 6, wherein themotion information included in the encoded stream decoded by the decoderis used for the processing for increasing the resolution by the imageresolution increasing unit; and the image corrector selects thedirection of a frequency component to be subjected to the low passfiltering by using the motion information included in the encoded streamdecoded by the decoder.
 11. An image storage device comprising: an imagereduction unit which reduces the size of an image included in an imagesignal and generates an image signal including the reduced image; astorage unit which stores the image signal having the reduced image; andan image resolution increasing unit which converts the image signal intoa high-resolution image by using an alias component and motioninformation that are included in the image signal stored in the storageunit.
 12. The image storage device according to claim 11, wherein theimage reduction unit performs processing for reducing the size of animage and low pass filtering on the image signal to cut a high frequencycomponent in the direction of the motion included in the image signal,wherein the cut-off frequency of the low pass filter for the imagesignal in the direction other than the direction of the motion is lowerthan the cut-off frequency of the low pass filter for the image signalin the direction of the motion.
 13. The image storage device accordingto claim 12, wherein the image resolution increasing unit performs theprocessing for increasing the resolution in the direction of the motionincluded in the image signal.
 14. The image storage device according toclaim 11 further comprising an image corrector which performs low passfiltering on an image signal including the high-resolution imagegenerated by the image resolution increasing unit, wherein the imagereduction unit reduces the image included in the image signal while thealias component remains in the image signal; the image resolutionincreasing unit performs the processing for increasing the resolution inthe direction of a motion included in the image signal; and the imagecorrector performs low pass filtering on the image signal to pass a lowfrequency component in a direction other than the direction of themotion.
 15. The image storage device according to claim 14 furthercomprising: an encoder which encodes the image signal generated by theimage reduction unit to generate an encoded stream and outputs theencoded stream to the storage unit that then stores the encoded stream;and a decoder which decodes the encoded stream stored in the storageunit; wherein the image corrector selects the direction of a frequencycomponent to be subjected to the low pass filtering by using the motioninformation included in the encoded stream decoded by the decoder.